xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark

Recent advancements in reference-free learned metrics for open-domain dialogue evaluation have been driven by the progress in pre-trained language models and the availability of dialogue data with high-quality human annotations. However, current studies predominantly concentrate on English dialogues, and the generalization of these metrics to other languages has not been fully examined. This is largely due to the absence of a multilingual dialogue evaluation benchmark. To address the issue, we introduce xDial-Eval, built on top of open-source English dialogue evaluation datasets. xDial-Eval includes 12 turn-level and 6 dialogue-level English datasets, comprising 14930 annotated turns and 8691 annotated dialogues respectively. The English dialogue data are extended to nine other languages with commercial machine translation systems. On xDial-Eval, we conduct comprehensive analyses of previous BERT-based metrics and the recently-emerged large language models. Lastly, we establish strong self-supervised and multilingual baselines. In terms of average Pearson correlations over all datasets and languages, the best baseline outperforms OpenAI's ChatGPT by absolute improvements of 6.5% and 4.6% at the turn and dialogue levels respectively, albeit with much fewer parameters. The data and code are publicly available at https://github.com/e0397123/xDial-Eval.Comment: Accepted to EMNLP-2023 Finding

Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4

The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic evaluation. Automatic evaluation of open-domain dialogue systems as an open challenge has been the center of the attention of many researchers. Despite the consistent efforts to improve automatic metrics' correlations with human evaluation, there have been very few attempts to assess their robustness over multiple domains and dimensions. Also, their focus is mainly on the English language. All of these challenges prompt the development of automatic evaluation metrics that are reliable in various domains, dimensions, and languages. This track in the 11th Dialogue System Technology Challenge (DSTC11) is part of the ongoing effort to promote robust and multilingual automatic evaluation metrics. This article describes the datasets and baselines provided to participants and discusses the submission and result details of the two proposed subtasks

Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration

Building document-grounded dialogue systems have received growing interest as documents convey a wealth of human knowledge and commonly exist in enterprises. Wherein, how to comprehend and retrieve information from documents is a challenging research problem. Previous work ignores the visual property of documents and treats them as plain text, resulting in incomplete modality. In this paper, we propose a Layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents (VRDs), so as to generate accurate responses in dialogue systems. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents, becoming the largest VRD-based information extraction dataset to the best of our knowledge. We also develop benchmark methods that extend the token-based language model to consider layout features like humans. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.Comment: Accepted to ACM Multimedia (MM) Industry Track 202

Genome-Wide Association Study Identified a Narrow Chromosome 1 Region Associated with Chicken Growth Traits

Chicken growth traits are important economic traits in broilers. A large number of studies are available on finding genetic factors affecting chicken growth. However, most of these studies identified chromosome regions containing putative quantitative trait loci and finding causal mutations is still a challenge. In this genome-wide association study (GWAS), we identified a narrow 1.5 Mb region (173.5–175 Mb) of chicken (Gallus gallus) chromosome (GGA) 1 to be strongly associated with chicken growth using 47,678 SNPs and 489 F2 chickens. The growth traits included aggregate body weight (BW) at 0–90 d of age measured weekly, biweekly average daily gains (ADG) derived from weekly body weight, and breast muscle weight (BMW), leg muscle weight (LMW) and wing weight (WW) at 90 d of age. Five SNPs in the 1.5 Mb KPNA3-FOXO1A region at GGA1 had the highest significant effects for all growth traits in this study, including a SNP at 8.9 Kb upstream of FOXO1A for BW at 22–48 d and 70 d, a SNP at 1.9 Kb downstream of FOXO1A for WW, a SNP at 20.9 Kb downstream of ENSGALG00000022732 for ADG at 29–42 d, a SNP in INTS6 for BW at 90 d, and a SNP in KPNA3 for BMW and LMW. The 1.5 Mb KPNA3-FOXO1A region contained two microRNA genes that could bind to messenger ribonucleic acid (mRNA) of IGF1, FOXO1A and KPNA3. It was further indicated that the 1.5 Mb GGA1 region had the strongest effects on chicken growth during 22–42 d

A fracture evaluation by acoustic logging technology in oil-based mud: A case from tight sandstone reservoirs in Keshen area of Kuqa Depression, Tarim Basin, NW China

To solve the problem of poor fracture identifying effect on electrical logging in oil-based mud, the application of acoustic logging to the quantitative characterisation of fractures is expanded from three aspects, namely, Stoneley waves, longitudinal and transverse waves and cross dipole acoustic waves, and a fracture logging evaluation model closely related to production capacity is established considering the radial extension characteristics of fractures. The Stoneley reflection coefficient is used to determine fractures locations to help detect fractures during normal micro-resistivity imaging logging. Based on the experiment on the relationship between fracture width and acoustic attenuation coefficient, empirical formulae for calculating fracture width have been established by primary wave and shear wave energy information considering the effect of porosity. The new parameters, including spectrum correlation coefficient and energy difference from cross dipole array acoustic logging data, can be used for fractures evaluation. The more developed the fractures are, the greater the energy difference becomes, and the smaller the spectrum correlation coefficient is, the higher the production is. The fracture effective evaluation parameters can be separated into two components, specified as the degree of fracture vertical opening and radial extension. Combining the conventional logging and array acoustic logging (including cross dipole array acoustic logging), a fracture radial extension evaluation model is presented closely related to productivity. Key words: tight sandstone reservoir, fracture evaluation, acoustic logging, oil-based mud, radial extension, Kuqa, Tarim Basi

Cement bond quality evaluation based on acoustic variable density logging

A new method of cement bond quality evaluation was proposed by combining numerical simulation and calibrated cased hole acoustic logging data. The effects of the cement channel angle and the quality of the second bond interface (the interface of cement with formation) on acoustic variable density logging data were analyzed. Based on the analysis result, a new cement bond evaluation standard was presented after revising the traditional CBL/VDL method. The axisymmetric acoustic field was simulated by real axis integral method, while the non-axisymmetric acoustic field was simulated by 2.5-D finite differential method. After comparing with the calibrated cased hole acoustic logging data, the research has the below results: the numerical simulation result matches with the calibrated well logging data very well and the new method is reliable; the amplitude of the first acoustic arrival in the case hole decreases as the angle of cement channel decreases, and the denser the cement is, the faster the amplitude of cased hole acoustic waveform decays; the lower limit of cement channel angle is around 45 degrees which can be detected by acoustic logging; the formation acoustic waveform is not easy to be detected in time domain, however it is easy to be detected in frequency domain, especially in limestone formation, the first arrival only can be detected when the annulus width of the second bond interface is small. According to the research result of the numerical simulation of cased hole acoustic field and acoustic variable density logging data, new evaluation criteria of cement bound quality were presented. Key words: cement bond quality, acoustic variable density logging, acoustic field in cased well, cement channel angel, interface bond inde

DKPLM: Decomposable Knowledge-Enhanced Pre-trained Language Model for Natural Language Understanding

Knowledge-Enhanced Pre-trained Language Models (KEPLMs) are pre-trained models with relation triples injecting from knowledge graphs to improve language understanding abilities.Experiments show that our model outperforms other KEPLMs significantly over zero-shot knowledge probing tasks and multiple knowledge-aware language understanding tasks. To guarantee effective knowledge injection, previous studies integrate models with knowledge encoders for representing knowledge retrieved from knowledge graphs. The operations for knowledge retrieval and encoding bring significant computational burdens, restricting the usage of such models in real-world applications that require high inference speed. In this paper, we propose a novel KEPLM named DKPLM that decomposes knowledge injection process of the pre-trained language models in pre-training, fine-tuning and inference stages, which facilitates the applications of KEPLMs in real-world scenarios. Specifically, we first detect knowledge-aware long-tail entities as the target for knowledge injection, enhancing the KEPLMs' semantic understanding abilities and avoiding injecting redundant information. The embeddings of long-tail entities are replaced by ``pseudo token representations'' formed by relevant knowledge triples. We further design the relational knowledge decoding task for pre-training to force the models to truly understand the injected knowledge by relation triple reconstruction. Experiments show that our model outperforms other KEPLMs significantly over zero-shot knowledge probing tasks and multiple knowledge-aware language understanding tasks. We further show that DKPLM has a higher inference speed than other competing models due to the decomposing mechanism

Experimental Study on Shear Wave Transmission in Fractured Media

Unconventional oil and gas reservoirs have broad exploration and development prospects. Fracture parameters and effectiveness evaluation are two of the key tasks for the evaluation of these types of reservoirs. Array acoustic logging can be used for fracture evaluation to compensate for the deficiencies of the image logging fracture evaluation method. Therefore, to develop acoustic logging evaluation methods as well as nondestructive testing methods for fractures, experiments were conducted to study the shear wave transmission in fractured media. Experiment data demonstrate a good correlation between the shear wave attenuation coefficient and fracture width, and the shear wave attenuation coefficients rise logarithmically with the increase in the fracture width for all models with different porosities and distinct dip angles of fractures. The shear wave attenuation coefficient changes relatively faster with the fracture width when the fracture width is within 250 μm. In addition, the shear wave attenuation is affected by the core porosity and fracture dip angle. When the fracture width is constant, the shear wave attenuation caused by the 0° fracture is relatively larger and is obviously greater than that of the fractures at other angles, which is consistent with the existing experimental results. The results of this study can be used to guide further research on amplitude compensation methods for sonic signal transmission in fractured media and fracture evaluation methods